ANR Project Involved

Participants: Amr Alyafi, Patrick Reignier.

Other Partners: UMR G-SCOP, UMR LIG (Persuasive Interaction, IIHM), CEA Liten, PACTE, Vesta Systems and Elithis.

Dates: Jan 2015 to Dec 2018

The ANR project Involved focuses on bringing solutions to building actors for upcoming challenges in energy management in residential buildings. The project explores a user centric energy management system, where user needs and tacit knowledge drive the search of solutions. These are calculated using a flexible energy model of the living areas. The system is personified by energy consultants with which building actors such as building owners, building managers, technical operators but also occupants, can interact with in order to co-define energy strategies, benefiting of both assets: tacit knowledge of human actors, and measurement with computation capabilities of calculators. Putting actors in the loop, i.e. making energy not only visible but also controllable is the needed step before large deployment of energy management solutions. It is proposed to develop interactive energy consultants for all the actors, which are energy management aided systems embedding models in order to support the decision making processes. MIRROR (interactive monitoring), WHAT-IF (interactive quantitative simulation), EXPLAIN (interactive qualitative simulation), SUGGEST-AND-ADJUST (interactive management) and RECOMMEND (interactive diagnosis) functionalities will be developed.

ANR Project CEEGE: Chess Expertise from Eye Gaze and Emotion

Participants: James Crowley, Dominique Vaufreydaz, Rafaellea Balzarini

Other Partners: Dept of NeuroCognition, CITEN, Bielefeld University

Dates: Jan 2016 to Dec 2019

CEEGE is a multidisciplinary scientific research project conducted by the Inria PRIMA team in cooperation with the Dept of Cognitive Neuroscience at the University of Bielefeld. The primary impacts will be improved scientific understanding in the disciplines of Computer Science and Cognitive NeuroScience. The aim of this project is to experimentally evaluate and compare current theories for mental modelling for problem solving and attention, as well as to refine and evaluate techniques for observing the physiological reactions of humans to situation that inspire pleasure, displeasure, arousal, dominance and fear.

In this project, we will observe the visual attention, physiological responses and mental states of subject with different levels of expertise solving classic chess problems, and participating in chess matches. We will observe chess players using eye-tracking, sustained and instantaneous face-expressions (micro-expressions), skin conductivity, blood flow (BVP), respiration, posture and other information extracted from audio-visual recordings and sensor readings of players. We will use the recorded information to estimate the mental constructs with which the players understand the game situation. Information from visual attention as well as physiological reactions will be used to determine and model the degree to which a player understands the game situation in terms of abstract configurations of chess pieces. This will provide a structured environment that we will use for experimental evaluation of current theories of mental modeling and emotional response during problem solving and social interaction.

The project is organized in three phases. During the first phase, we will observe individual players of different levels of chess expertise solving known chess problems. We will correlate scan-path from eye tracking and other information about visual attention to established configurations of pieces and known solutions to chess problems. This will allow us to construct a labeled corpus of chess play that can be used to evaluate competing techniques for estimating mental models and physiological responses. In a second phase, we will observe the attention and face expressions of pairs of players of different levels of chess ability during game play. In particular, we will seek to annotate and segment recordings with respect to the difficulty of the game situation as well as situations that elicit particularly strong physiological reactions. In the final phase, we will use these recordings to evaluate the effectiveness of competing techniques for mental modeling and observation of emotions in terms of their abilities to predict the chess abilities of players, game outcomes and individual moves and player self reports. Results of our work will be published in scientific conferences and journals concerned with cognitive science and cognitive neuroscience as well as computer vision, multimodal interaction, affective computing and pervasive computing. Possible applications include construction of systems that can monitor the cognitive abilities and emotional reactions of users of interactive systems to provide assistance that is appropriate but not excessive, companion systems that can aid with active healthy ageing, and tutoring systems that can assist users in developing skills in a variety of domains including chess.